An important task when operating a network is to forward and distribute data, as for example performed by routers, NATs (network address translators) and switches. Such and similar devices, referred to as network devices hereinafter, can be separate hardware entities or can be dedicated general purpose computers.
Network devices are often designed to export information on the network traffic they process. Such information can, for example, be used for billing purposes or to monitor traffic for load balancing or to detect malicious traffic, e.g. a denial of service attack. Network traffic usually consists of data packets, several of which make up a data entity, for example a data file. A single transaction might comprise a plurality of data entities to be exchanged between two or more units within the network, for example servers as data sources and clients as destinations. Information on network traffic can be presented on many different levels, for example on the packet level or the transaction level. Information on the packet level can usually be acquired from a Management Information Base (MIB) maintained by the network device. A system for monitoring the performance of a network based on information contained in MIBs is for example disclosed in US 2004/0054680 A1.
A commonly used level for presenting information on network traffic that allows a deeper and more sophisticated analysis of the network traffic is based on network flows. A network flow may be defined as a unidirectional stream of packets from a certain source to a certain destination. A commonly used data format in which network flow information can be exported is defined by Cisco's flow profiling system NetFlow, as for example described in the manual “Cisco IOS Release 12.0 (5) T”. An open, general and flexible standard called IPFIX (Internet Protocol Flow Information eXport) is being standardized by an IETF (Internet Engineering Task Force) workgroup.
For a network device to be able to export information on network traffic based on network flows, the network device usually keeps track of flows using a flow state table, sometimes also called flow table or flow cache. Each flow is represented by an entry in the flow state table, where, for example, the number of transmitted packets or transmitted bytes belonging to each flow is recorded. Upon detection of a packet stating a termination of the flow or after a certain period of time has run out, the flow information is exported and subsequently the entry deleted from the flow state table.
Problems can occur under increasing traffic load. As a result of increasing traffic load the flow state table might become so large that updating the flow state table might negatively affect the network devices' performance for processing network traffic. To reduce this effect, an associative memory device can be used to identify network flow information and maintain the flow state table as disclosed by U.S. Pat. No. 6,871,265 B1. Under more traffic, the flow state table might further exceed its maximum capacity so that no further flow information can be gathered. Also, the number of exported netflows can get so high that the additional network traffic negatively affects the network performance.
According, for example, to Cisco's NetFlow system, a solution to this problem is to statistically sample or filter incoming packets to decrease the number of packets considered in the flow state table. Another solution is to use a wider, more general definition of what constitutes a network flow in order to aggregate information. Sampling, filtering and aggregation policies are used to manually configure network devices to utilize one of these or a combination of these techniques. U.S. Pat. No. 6,446,200 B1 and U.S. Pat. No. 6,751,663 B1 disclose a system in which a data collector layer receives network flow information from network devices and sends it to a flow aggregation layer that aggregates received records and stores the aggregated information. That way, aggregation can be performed on packets processed by different network devices, which reduces the amount of finally stored aggregated information even more than aggregation performed in each network device, but does not reduce network traffic due to the transmission of network flow information from the data collector layer to the flow aggregation layer. A drawback of all mentioned techniques is that they might compromise fidelity of the network flow information, in particular if not well tuned to the actual kind of network traffic.
It is therefore a challenge to provide a method, a device and a computer program product for configuring a network flow information exporting device to export network flow information with better fidelity and in a less resource-intensive manner. It is further a challenge to provide a system for processing network traffic and exporting network flow information more effectively.